Semantic Instance Annotation of Street Scenes by 3D to 2D Label Transfer

Semantic annotations are vital for training models for object recognition, semantic segmentation or scene understanding. Unfortunately, pixelwise annotation of images at very large scale is labor-intensive and only little labeled data is available, particularly at instance level and for street scenes. In this paper, we propose to tackle this problem by lifting the semantic instance labeling task from 2D into 3D. Given reconstructions from stereo or laser data, we annotate static 3D scene elements with rough bounding primitives and develop a model which transfers this information into the image domain. We leverage our method to obtain 2D labels for a novel suburban video dataset which we have collected, resulting in 400k semantic and instance image annotations. A comparison of our method to state-of-the-art label transfer baselines reveals that 3D information enables more efficient annotation while at the same time resulting in improved accuracy and time-coherent labels.Comment: 10 pages in Conference on Computer Vision and Pattern Recognition (CVPR), 201

Dynamic region of interest transcoding for multipoint video conferencing

This paper presents a region of interest transcoding scheme for multipoint video conferencing to enhance the visual quality. In a multipoint videoconference, usually there are only one or two active conferees at one time which are the regions of interest to the other conferees involved. We propose a Dynamic Sub-Window Skipping (DSWS) scheme to firstly identify the active participants from the multiple incoming encoded video streams by calculating the motion activity of each sub-window, and secondly reduce the frame-rates of the motion inactive participants by skipping these less-important subwindows. The bits saved by the skipping operation are reallocated to the active sub-windows to enhance the regions of interest. We also propose a low-complexity scheme to compose and trace the unavailable motion vectors with a good accuracy in the dropped inactive sub-windows after performing the DSWS. Simulation results show that the proposed methods not only significantly improve the visual quality on the active subwindows without introducing serious visual quality degradation in the inactive ones, but also reduce the computational complexity and avoid whole-frame skipping. Moreover, the proposed algorithm is fully compatible with the H.263 video coding standard. 1